Papers with uncertainty estimates
Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression (2022.tacl-1)
Copied to clipboard
| Challenge: | State-of-the-art classification and regression models are often not well calibrated and can be inaccurate. |
| Approach: | They quantify calibration of pre- trained language models for text regression . they apply uncertainty estimates to augment training data in low-resource domains . |
| Outcome: | The proposed model calibrations improve performance and generalizability in low-resource settings. |
Enhancing Zero-shot Chain of Thought Prompting via Uncertainty-Guided Strategy Selection (2025.coling-main)
Copied to clipboard
Shanu Kumar, Saish Mendke, Karody Lubna Abdul Rahman, Santosh Kurasa, Parag Agrawal, Sandipan Dandapat
| Challenge: | Existing methods for chain-of-thought (CoT) prompting are limited by handcrafted demonstrations and trigger phrases are prone to inaccuracies. |
| Approach: | They propose a method that generates rationales using a trigger phrase to select effective demonstrations without accessing model parameters. |
| Outcome: | The proposed method outperforms existing methods across four reasoning benchmarks and is robust and scalable. |
UNComp: Can Matrix Entropy Uncover Sparsity? — A Compressor Design from an Uncertainty-Aware Perspective (2025.emnlp-main)
Copied to clipboard
Jing Xiong, Jianghan Shen, Fanghua Ye, Chaofan Tao, Zhongwei Wan, Jianqiao Lu, Xun Wu, Chuanyang Zheng, Zhijiang Guo, Min Yang, Lingpeng Kong, Ngai Wong
| Challenge: | Deploying large language models (LLMs) for long-context inference remains challenging due to their substantial memory and computational demands. |
| Approach: | They propose an uncertainty-aware framework that leverages truncated matrix entropy to identify areas of low information content. |
| Outcome: | The proposed framework reduces the KV cache size to 4.74% of the original and achieves a 6% speedup. |
Predicate-Conditional Conformalized Answer Sets for Knowledge Graph Embeddings (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods provide probabilistic guarantees over a reference set of queries and answers, but they fail to identify when the answers to a query are uncertain. |
| Approach: | They propose a method that approximates predicate-conditional coverage guarantees while maintaining compact prediction sets. |
| Outcome: | The proposed method provides predicate-conditional coverage guarantees while maintaining compact prediction sets. |
SUNAR: Semantic Uncertainty based Neighborhood Aware Retrieval for Complex QA (2025.naacl-long)
Copied to clipboard
| Challenge: | Open-domain complex question-answering systems face challenges in retrieving and reasoning over information that addresses multifaceted queries. |
| Approach: | They propose a method that leverages large language models to guide a Neighborhood Aware Retrieval process. |
| Outcome: | The proposed approach outperforms retrieve-and-reason baselines on two complex QA datasets. |
Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study (D18-1)
Copied to clipboard
| Challenge: | Existing studies on Active Learning (AL) for natural language processing have limited data requirements. |
| Approach: | They propose a Bayesian active learning approach that reduces deep learning's data dependence by comparing models and acquisition functions. |
| Outcome: | The proposed approach outperforms i.i.d. baselines and is more efficient than other approaches. |
AURORA: Neuro-Symbolic Continual Indexing for Evolving RAG Systems (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) systems depend on non-parametric indices to access external knowledge. |
| Approach: | They propose a framework for adapting retrieval indices under distribution shift . AURORA decouples discrete index structure from continuous metric representations . it recovers +26.9% Recall@10 on novel topics compared to static baselines compared with static baseline . |
| Outcome: | AURORA decouples discrete index structure from continuous metric representations . it recovers +26.9% Recall@10 on novel topics while adapting significantly faster than full retraining. |
Estimating predictive uncertainty for rumour verification models (2020.acl-main)
Copied to clipboard
| Challenge: | Inability to correctly resolve rumours can have harmful real-world consequences. |
| Approach: | They propose a method for incorporating model and data uncertainty estimates into natural language processing models for automatic rumour verification. |
| Outcome: | The proposed methods filter out erroneous model predictions and prioritise them for a human fact-checker. |
Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Decoding strategies affect the probability distribution underlying the output of a language model and can therefore affect both generation quality and uncertainty. |
| Approach: | They investigate the impact of decoding strategies on uncertainty estimation in large language models . |
| Outcome: | The proposed methods improve the uncertainty estimation of large language models by reducing repetition. |
Simple Yet Effective: An Information-Theoretic Approach to Multi-LLM Uncertainty Quantification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work on calibration and uncertainty quantification focuses on individual models, overlooking the potential of model diversity. |
| Approach: | They propose a method that uses Jensen-Shannon Divergence to identify and aggregate well-calibrated subsets of large language models (LLMs) to improve calibration. |
| Outcome: | The proposed method improves accuracy on binary prediction tasks compared to single-model and naive ensemble baselines. |
CURA: Clinical Uncertainty Risk Alignment for Language Model–Based Risk Prediction (2026.acl-long)
Copied to clipboard
| Challenge: | Clinical language models (LMs) are increasingly applied to support clinical risk prediction from free-text notes, yet their uncertainty estimates are poorly calibrated and clinically unreliable. |
| Approach: | They propose a framework that aligns clinical LM-based risk estimates and uncertainty with individual error likelihoods and cohort-level ambiguities. |
| Outcome: | The proposed framework improves accuracy on clinical risk prediction tasks without compromising discrimination. |
Uncertainty-Aware Test-Time Search for Optimization Problem Solving (2026.acl-long)
Copied to clipboard
Linlin Yu, Xujiang Zhao, Dong Li, Yanchi Liu, Wei Cheng, Zhengzhang Chen, Chen Zhao, Feng Chen, Haifeng Chen
| Challenge: | Language model hallucinations and limited availability of labeled datasets often result in misaligned formulations, code errors and feasibility failures. |
| Approach: | They propose a Monte Carlo Tree Search framework that automates optimization problems from natural language descriptions with efficiency and reliability. |
| Outcome: | The proposed framework achieves state-of-the-art solution accuracy and reduces token usage. |